Add public API governance instruction evaluation - #68698
Draft
PureWeen wants to merge 6 commits into
Draft
Conversation
PureWeen
changed the base branch from
main
to
pureween-public-api-instructions
August 21, 2026 16:38
PureWeen
force-pushed
the
pureween-public-api-instruction-eval
branch
from
August 25, 2026 15:52
c80cac4 to
ba87f27
Compare
Youssef1313
added
the
area-infrastructure
Contributor
|
Hey @dotnet/aspnet-build, looks like this PR is something you want to take a look at. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Align the specialized suite with the RTM approval boundary and exercise the api-review skill handoff introduced by #68959. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 17c247c0-c8ff-497c-82ac-ef81dd1ba327
PureWeen
force-pushed
the
pureween-public-api-instruction-eval
branch
from
September 7, 2026 13:32
ba87f27 to
27622b9
Compare
Set both the evaluated agent and judge defaults to GPT-5.4 so the specialized suite does not use Anthropic models. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 17c247c0-c8ff-497c-82ac-ef81dd1ba327
Strengthen the specialized graders so responses must name the api-review skill, proposal labels, PublicAPI.Unshipped distinction, and revised ref-assembly proposal workflow. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 17c247c0-c8ff-497c-82ac-ef81dd1ba327
Set both the evaluated agent and judge defaults to GPT-5.6 Luna for the specialized public API governance suite. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 17c247c0-c8ff-497c-82ac-ef81dd1ba327
Load the complete api-review skill through Vally's environment.skills, require its activation for PR authoring, and remove brittle proximity matching from otherwise complete responses. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 17c247c0-c8ff-497c-82ac-ef81dd1ba327
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
api-reviewskill and creates a separate linked proposal issueapi-approvedcovers the final implemented shape before RTM inclusion and that changed approved shapes return to API review.github/copilot-instructions.mdthroughenvironment.filesand the complete.github/skills/api-reviewdirectory throughenvironment.skillsThis is the remaining open layer of native stack #68699 after #68168 merged. It follows the specialized-suite conventions under
eng/skill-evalsand adds no Vally onboarding infrastructure.Validation
pwsh -NoProfile -File ./eng/skill-evals/run.ps1 Testpwsh -NoProfile -File ./eng/skill-evals/run.ps1 Validate -Eval eng/skill-evals/copilot-instructions/public-api-governance.vally.yamlapi-reviewskillgit diff --check